Regularized Color Demosaicing via Luminance Approximation

In single-sensor digital imaging a color filter array, that is overlaid onto the image sensor, makes color images possible. Incident light rays become band-limited and each sensor element captures either red, green or blue light. Interpolating the missing two color components for each pixel location is known as demosaicing. This paper proposes to firstly derive an estimated luminance image by low-pass filtering the original mosaiced sensor image. In a second step a deconvolution technique re-sharpenes the blurred luminance approximation, so that it has the same spatial resolution as the original but bandpassed sensor image. Using the high-resolution luminance approximation the partial RGB colors from the mosaiced sensor image are transformed into a different color space that is more suitable for color interpolation. The new color space consists of least correlated color data, so that intra-channel interpolation errors have a reduced impact on inter-channel alignment, and therefore result into less prominent interpolation artifacts. Demosaicing is performed on the transformed color data separately for each plane, whereby again the luminance approximation, which encodes the aligned gradient direction of all color channels, regularizes the bilinear interpolation. Finally, the result is remapped into the RGB color space to obtain the demosaiced color image. Additionally, correlated multi-channel anisotropic diffusion is applied onto the demosaiced color image to further reduce interpolation artifacts and enable denoising. The proposed algorithm is evaluated and it is concluded that although the image formation model could be verified its performance heavily depends on the quality of the luminance approximation, i.e. the deconvolution method. Problem Outline Because pixel elements of digital image sensors are only capable of sensing incident light photons, but cannot distinguish them by their wavelength, a color filter array (CFA) may be used to make the photo-detector array sensitive to color. A filter layer with an alternating mask of red, green and blue filters is placed on top of the image sensor, thereby band-limiting each sensor element according to some regular mosaic pattern. The most widely chosen pattern is the Bayer filter mosaic [3]. In this pattern half of the pixels are green, whereas each red and blue pixels cover 25 percent of the sensor. A first row of alternating blue and green pixels is followed by a second row of alternating green and red pixels. The Bayer pattern effectively reduces the spatial sampling resolution of the image sensor in favor of additional mul0 Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than IS&T must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Johannes Herwig and Josef Pauli, ”Regularized Color Demosaicing via Luminance Approximation”, CGIV 2010/MCS’10 5th European Conference on Colour in Graphics, Imaging, and Vision / 12th International Symposium on Multispectral Colour Science, Society for Imaging Science and Technology, pp. 62-69, 2010. tispectral sampling. The aim of an interpolation algorithm is to recover lost spatial resolution due to the filter mosaic, thereby preserving high frequency information. Especially, one has to avoid introducing visual artifacts like blurred edges, aliasing and false colors, which are mainly the result of interpolating across edges, falsly anticipated maximum intensity gradient direction within a color channel [5], and misregistration in color balancing [17], respectively. Another well known artifact in this domain is the so-called zippering effect, that is a pattern of alternating colors, that occurs in otherwise homogeneous regions or at sharp edges [11].